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Abstract 



The system mentioned in the title belongs to the family of the so-called 
massively multi- player online social games (MMOSG). It features a scor- 
ing system for the elements of the game that is prone to herding effects. 
We analyze in detail its stationary regime in the thermodynamic limit, 
when the nmnber of players tends to infinity. In particular, for some 
classes of input sequences and selection policies, we provide necessary 
and sufficient conditions for the existence of a complete meanfield-like 
measure, showing off an interesting condensation phenomenon. 

1 Description of the basic model 

Ma Micro Planete j6] is a geolocalized massively multi-player online social 
game which entices players to use sustainable means of transport. At the 
heart of the game is a community-driven creation of a set of points of interest 
(POIs): interesting places, events, or even traffic incidents. The popularity 
of the POIs is assessed by scores. At his turn, the player is presented with 
a number of POIs to visit in his neighborhood, which have been selected 
through a mechanism that favors already popular ones. Then the selected 
POI has its score increased by a random value. It is not difficult to see that 
there is a risk of herding behavior, that would concentrate all the attention 
on a few places only. The aim of this paper is to present some results about 
the POIs dynamics by means of a probabilistic model, and in particular to 
determine when a stationary solution exists. 

Assume there are players in the game and let ni{t; N) be the number 
of POIs having an integer score i at time t. The state of the system at time 
t can be described by the infinite sequence 
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R{t; N) 
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[ni{t;N),...,ni{t;N),...]. 



N 



The dynamics of the model is 

• POIs are created according to a Poisson process of rate XN. Each new 
POI is granted an initial score k with probability (pk,k > 1, where 
{Vk,k > 1} form a proper probability distribution. 

• For any i > 1, each POI with score i decreases its score by 1 after a 
random time exponentially distributed at rate /x, where /x is a strictly 
positive constant. 

• The N players are all independent and visit POIs at instants forming a 
global Poisson process with rate aN. Upon being visited, a POI sees 
its score increased by j > 1 with probability 6j, independently of any 
other event, where the sequence {9j,j > 1} forms a proper probability 
distribution. We shall implicitly take = 0, which is in no way a 
restriction, as it is simply tantamount to modifying the parameter a. 

• When he decides to play, at the instant of a Poisson process of param- 
eter aN, a player selects a POI having score i > 1 with probability 
Tri{R{t; N)), written as a functional of the state of the system, with the 
normalizing condition 

i>i 

In accordance with the description of the game, we shall essentially 
consider selection policies such that the probability tt of selecting a POI, 
given its score i, is a non-decreasing function of i. 

The following typical scoring rules are used in Ma Micro Planete. 

• Each player starts a journey at rate a and selects a POI to visit with a 
probability proportional to its current score. 

— with probability p, the player adds new information to the POI, 
the score of which is increased by 15 points; 

— otherwise, the POI is merely visited and its score gains 5 points; 
this happens with probability g == 1 — p. 

• Each player creates POIs at rate A and their initial score is set to 50. 

• The POIs decrease their scores by 1 at a rate /x = 3 per day. 

• A POI disappears as soon as its score becomes zero or negative. 

• The values of a, p and A can only be measured empirically from concrete 
game statistics. 
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A way of describing this model might be in terms of queues. Consider 
a possibly infinite array of queues M/M/1 queues with service rate /i. New 
queues are created at rate A and batches of customers join an existing queue 
with score i > 1 at rate a7ri(R(t; N)). Such systems have already been 
described, for example when the customers join the shortest of two queues [^. 
The situation described in this paper is unusual due to the bias towards 
selecting the longest queue. This only makes sense because there is no cost 
associated to visiting a highly popular POI, whereas joining a long queue 
is expensive in terms of waiting time. A similar situation [8j occurs when 
customers, unsure of the quality of two restaurants, choose the busier one. 

The model can have ramifications with other areas of interest (e.g., 
dynamics of populations, social networks, chemical or energy networks in 
physics) . 

In this work, we examine the system in the so-called thermodynamic limit, 
as — )• cx), the existence of which is proved in Section |2] under general 
conditions, where the probability of increasing the score of a visited POI is a 
function of the state of the system. Section [3] is devoted to the stationary 
regime, for which complete (but surprising!) answers are given for various 
input sequences and selection policies, and the existence of possible phase 
transition phenomena is enlightened. Section |4] presents miscellaneous simple 
limit cases. The final Section [5] outlines a possible method of solution for 
general input sequences, and also discusses a brief list of unsolved questions. 

2 The t her mo dynamical hmit 

To capture information on the behavior of the system under various game 
policies, we proceed by scaling and analyze the thermodynamical limit, when 
the number N of players becomes large. In mathematical terms, we consider 
the system when N — >• oo, for any fixed t. 

2.1 Notation 

It will be convenient to gather here most of the notational material concerning 
input sequences and mathematical symbols. 

m £ = = [yj^, . . . , T/j, . . .], G Q}, where is the set of positive 
rational numbers. MJ^ being the set of non-negative real numbers, ej 
will stand for the j-th unit vector of MP^. We shall also denote by 

L G the subset of bounded sequences ||y|| == supj \ yi\ < M < oo. 

• E being an arbitrary metric space, B{E) is the space of bounded real- 
valued Borel measurable functions on E, C{E) C 13{E) is the Banach 
subspace of bounded continuous functions, and Ci{E) is the subspace of 
functions with uniformly bounded continuous derivatives of first order. 
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We shall need the two following subspaces 



C2 = |/GCi(L):j; 

.7>1 



< c 



/gCi(L) : sup 



dy"^ dyl dyjdyk 



C, H being arbitrary positive constants. 

• For any complete separable metric space S*, let L)5[0,oo] denote the 
space of right continuous functions / : [0, oo] — )• S with left limits, 
endowed with the Skorokhod topology. 

2.2 The main limit theorem 

Clearly R{t;N) is a continuous time Markov chain, and its generator C^', 
for / S B{L,), is the operator given by. 



G^-^f{y) = XN^^, 
+ I^N^yj 



f{y + ^]-fiy) 



f [y + 



N N 



f{y) 



i>l 



f y-^ + 



i+j 



N N 



f{y) 



(2.1) 



Throughout this study, the system will be subject to the following weak 
assumption. 



Assumption (L) The probability distribution {7rj(r),i > 1} is afunctional 
supposed to be twice differentiable with respect to r, for all r G L. 

The above assumption (L) holds in particular for the forthcoming policy 
denoted by Pg, which will be analyzed in Section [3j 



■Ki{r) 



aiVi 



where the at form a sequence of positive numbers. 

Then, with the notation 9q = t^o{.) = 0, the following proposition holds. 

Theorem 2.1. Suppose that, as N ^ oo, i?(0; A^) converges in distribution to 
i?(0) G L. Then the Markov process R{t; N) converges also weakly in D[0, oo] 
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to a deterministic dynamical system G MJ^, which evolves according to 
the following infinite system of nonlinear differential equations 



drijt) 
dt 



Xifi + /i[ri+i(t) - nit)] +a[Y^ ejiTi^jifit)) - TTi{f{t))] , (2.2) 



j=0 



where 'Ki{f{t)) denotes the probability that the currently visited POI has score i. 
Moreover, under Assumption (L) and for any given finite initial condition 
r(0) G L, the system (2.2) admits a unique solution, for all t >0. 

Proof. The method to solve this weak convergence problem is somehow 
classical (see for example the footsteps of [T]). First, a second order Taylor's 
expansion in (2.1) yields immediately, for any / S C2(L), 

1 



G(-'/(y) = i3/(y) + 



N 



where B is the operator with domain Ci(L) satisfying 



i>i i>i 



df df 

9yj 



df df 



(2.3) 



Under Assumption (L), which is stronger than the famous Lipschitz 
condition, it is well-known that the system (2.2) admits a unique positive 
solution, for any initial finite condition r(0) and all finite t. Then for / G C(L) 
one can define a one-parameter family {T{t),t > 0}, such that 

r(t)/(f(o)) = /(f(t)). 

It is not difficult to see that {T(t),t > 0} is indeed a strongly continuous 
contraction semi-group, with generator, say G, which is closed by Theorem 



A.l Now the two following properties can be checked at once. 

• For ah t, r{t) £ 'H2. 

• Taking / G C2, then clearl y T{ t)f{f{0)) G 7^2- Since C2 is dense in 
C(L), it follows by Theorem A. 3 that "^2 is a core for G. Hence G = B. 



The conclusion of the proof is a direct application of Theorem A. 4 together 
with standard results on differential systems, see e.g. j5]. □ 
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2.3 Some analytic facts 

Let, for \z\ < 1, 



i=l 



viz) = J2 



1=1 

oo 



ifiZ 



e{z) = Y,Oiz\ 



i=l 



i=0 



In this respect, the model of Ma Micro Planete presented in Section [T] 
corresponds to 

e{z) = pz'^^ + qz^ , (/7(z)=2;^°. 



Taking the summation over i in system (2.2), we get immediately 
dr(t, z) 



1 



1 



r{t,z) = \^{z) - ^iri{t) + a[e{z) - l]7r(t,z). (2.4) 



The following proposition ensures in particular, for the class of policies 
considered in Section 2.2 the boundedness of the Cesaro limit of r(t, 1) as 
i — )• oo, which, from its mathematical definition, takes into account the POIs 
possibly going to infinity. 

Proposition 2.2. When {9k, k >1} is a proper probability distribution, so 
that 9{1) = 1, we have 



lim - 

t— >oo t 







ri{t)dt=-. 



(2.5) 



Moreover, if 9'{l).ip'{l) < oo, and for any policy satisfying Assumption (L), 
1 



lim 

>oo t 



V(t,l)dt=^^^^ill±Ml) < «^^(1) + A(^^(1) ^ 



where 



TT = lim STT* ( s, — — — 
s-S>0 V /X + s 



(2.6) 



< 1. 



Proof. For any arbitrary positive function /, let /* denote its ordinary Laplace 
transform 



f*{s)'^ e-''f{t)dt, ^{s)>0. 
Jo 



Then, by a simple algebra carried out on equation (2.4), we obtain 

'X(piz) 



r*{s,z) 



1 



+ a{9{z) - l)7r*(s, z) - /ir^s) + r(0, z) 



(2.7) 
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Furthermore, applying Cauchy's formula 

ri(s)- 



2i7r Jc 



in equation (2.7), where C stands for the unit circle centered at the origin, 
we get 



1 



\ip{y)dy 



+ 



2m Jc sy [(/x + s)y - ^] 

1 f [ a{e{y) - l)7r* {s, y) - K (g) + ^(0, v)] 
y[{^l + s)y - ii] 



2iTr 



which in turn, by a repeated (and elementary) application of Cauchy's residue 
theorem, allows to extract 



rlis) 



jJLS /i 



fJ. + s 



IT S 



fi + s/ fl \ fJ, + s 



. (2.8) 



Hence, instantiating the latter expression of rj(s) in (2.7), we obtain finally 
1 

+ a{9{z) - l)7r*(s, z) + a(^l- 



r*{s, z) 



vr I s, 



+ s 



2.9) 



+ r(0,.)-.(0,^) 



Take now s > and let z — t- 1_ by real positive values in (2.9). This yields 



r*{s,l) 



1 



1-99 



/U + s 



+ a 1 



vr s 



(2.10) 



where we have used the fact that vr*(s, 1) = 1/s, since 7r(t, 1) = 1, Vt > 0. 

Moreover, vr* ^s, < t^*{s, 1) = ^, for any s > 0, and we are now in a 

position to apply a Tauberian-type theorem for positive functions, see e.g. j3], 
just letting s — )• by positive values in (2.10), so that 

ft 



1 r 

limsr*(s, 1) = lim - / r(t, l)dt 
s^O t->oo t Jq 



lim 

s-S>0 



1-^ 



/i + s 



s 



/i + s 



S7r*(s,l) 



a7r6''(l) + X^'{1) 
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which is exactly the asserted equahty (2.6). It is worth noting that for the 
moment there is no need to assume the finiteness of the derivatives ^'(1) and 
(/9'(1), but this win be made more precise in Section [3] 



As for (2.5), it is readily obtained from ( |2.8[ ), using the inequality 



ST^* \S, < 1) and again the above Tauberian theorem. The proof of the 

proposition is concluded. □ 



3 System behavior as t — oo 

In the sequel, we shall focus our attention solely on the deterministic dy- 



namical system f{t) obtained in Theorem 2.1 Of special interest will be the 
non-degenerate stationary limits 

r = lim r{t), with rj(t) — )• rj,Vi > 1, 

and 

lim lim 7r(f, z) = 7f. (3-1) 

Indeed, under Assumption (L), for a given set of parameters, a unique 
stationary regime of the system exists, but possibly degenerate in the sense 
that 7f < 1. This follows from the general theory of differential systems (see 
e.g., 0). Note that this fact is not easy to prove by a probabilistic argument. 



since, due to the normalizing condition, system (2.2) cannot be interpreted 
as forward Kolmogorov's equations for a special birth and death process, 
where r{t, z)/r{t, 1) would represent the probability distribution function of 
the score of an arbitrary POL 

Throughout the rest of the paper, we shall speak of ergodicity, with a 
slight abuse of language: this will always refer to a finite stationary dynamical 
system satisfying the conditions 

r(l) = ^ri<oo, 7f=l. (3.2) 

i>l 

3.1 Global conservation equations 

Define the generating functions 



oo oo 

, \ dcf ' 

riz) 



i=l i=l 

with the shortened notation tt{z) = 7r(z; r), and 

^Y^=Te„z", l^^='^ci,„,". (3.3) 

n>0 n>0 

Two characteristic values are of special interest, namely 
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r(l) = ^j>]^ ri, the mean number of POIs per player; 
ri, since ^uri is the global death rate of POIs per player. 



By (2.4), one sees easily that r{z) satisfies the functional equation 

a [1 - B{z)] 7r{z) + - z-^]r{z) = Xip{z) - firi, (3.4) 
from which can be deduced the following corollary, which is the direct 



continuation of Proposition 2.2 
Corollary 3.1. 

1. We have always lim(l — z)r(z) = and ri = —. 

Z-5-1 fj, 

2. If in addition 9'{l).ip'(l) < oo, then 



rizl^aeizMz)^>Mzl^ VN < 1, (3.5) 



Proof. Immediate from Proposition 2.2 together with definition (3.3). □ 



Assumption (F) In the rest of the paper, unless otherwise explicitly men- 
tioned, the product 0'{l).ip'{l) < oo will always be assumed to be finite. 

In the next sections, we examine in detail the effect of specific choices 
of the policy 7r(.) on the behavior of the system. One of main goals of the 
study is to find out the necessary and sufficient conditions for the system to 



be ergodic in the sense given by the conditions (3.2). 



3.2 Selection of POIs according to their scores 

In this section, we consider a set of policies where the selection of a POI 
depends solely on its score. Letting each score i > 1 be associated with a 
positive weight Oj, then we recover the policy announced in Assumption 



(L) of Section 2.2 namely 



i=i 

In the context of Ma Micro Planete, it seems quite realistic to assume that 
the Cj's are increasing with i, since POIs with high scores should be more 
attractive. In the sequel, we shall mainly consider this situation of herding 
effect. 
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From equation (3.5), we get the immediate recm'sion, valid Vn > 0, 

n . 

Since the Oj's are assumed to form an increasing sequence, there exists 
liuii-^oo CLi- When this hmit is not finite, the foUowing simple interesting 
proposition holds. 

Proposition 3.2. // limj_j>oo = oo, then the system is non-ergodic. 



Proof. The relation (3.8) leads to the simple bound 



a a 

rn+l > —f-^anrn'^O = —^anVn, 

/iiv fj,K 

so that no finite non-zero solution to the system can exist when the Oj's are 
unbounded. □ 



Consequently the only interesting case is when {ai,i > 1} is a non- 
decreasing sequence tending to a finite limit, which ad libitum can be taken 
equal to 1, since the Oj's are defined up to a constant. 

3.3 When scores of selected POIs are exactly increased by 
one 



As it will appear in Sections |3.5|and 5.1 a complete solution of the non-linear 



equation (3.8 ) for a general 9{z) seems to be technically almost untractable. In 
this section, in order to get ideas about the general behavior of the system, we 
analyze in detail the simple (but nonetheless not elementary!) case 6{z) = z, 



for which equation (3.8) becomes the following first order recursive sequence 



a X 

r-n+i = —r^anVn + -5>„. (3.9) 
A simple algebra carried out on this last equation leads to the formula 

rn,r = -An^j{-) , n>0, (3.10) 

where 

n 
i=l 
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The constant K can then be recovered as 



i>l 



,00 I ^ ■ ■ 



j=0^jl^Xy i=j 



a Y 



a 



E^^ 



f— V 

j=0^j\^,K) i=j+l ^ 

which leads to the following implicit equation 

allowing to determine K, after having set 

00 



(3.11) 



k=l 



E 



(3.12) 



A. 

3=0 ^ 



yk > 1. 



So, the problem of showing the existence of a strictly positive and finite 
constant K is exactly tantamount to finding a strictly positive real number 
X, such that 

00 

F{x) = ^UkX^. 



A 



fe=i 



Hence, in order to get effective ergodicity conditions, one has to check the 
lower and upper bounds of F{x), which is an increasing function of x, for 
X > 0. We note first that F{x) is finite for small enough |x| if, and only if. 



ui = ^^Oj+i^j < 00. 
i=0 



(3.13) 



But, ex hypothesis, A^+i/Ak = ak increases to 1 from below, so that Uk is a 
monotone decreasing sequence and inequality (3.13) is plainly equivalent to 

(3.14) 



= E^i < 



As for for the upper bound, we have the immediate inequality 

Uk < Vfc > 1, 
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and hence the radius of convergence of F{z) is exactly equal to 1. Now, in 
order to decide whether the consistency relation (3.11) admits a solution, the 
key point will be to analyze the quantity 

M = lim F{x). 

X—>-l- 

There are two possibilities. 

(i) If M is infinite, then for any value of A, a, fi, it is possible to find a 
unique admissible finite K, and we shall say that the system is ergodic. 

(ii) Conversely, if M < oo, then it will act as a limiting value for a/ A, and 
hence a non trivial ergodicity condition takes place. 

The situation is summarized in the following brief statement. 

Theorem 3.3. Assume that lim Oj 1. 

(i) The system is never ergodic when (/5'(1) = oo. 
(ii) When ^'(1) < oo, the system is ergodic if, and only if, 

k=l j=0 ^ 

where the r.h.s. of the inequation above may he infinite. 

It is interesting to note a few not so intuitive facts related to (3.15 ): firstly, 
this condition does not depend on the value of ^, as long as /x > obviously. 
Secondly, the ergodicity range increases when the initial score increases in 
distribution — that is, when the $j's increase. 

To propose a somehow concrete classification, let us concentrate for the 
remainder of this section on the reasonably general set of sequences satisfying, 
for some strictly positive constants 7 and z/, 

• ai > 0, 

• the Oj's form an increasing sequence. 

The following proposition describes the phase transition phenomenon that 
occurs when the parameters vary. 

Proposition 3.4. Assume the sequence ai satisfies the above conditions, and 
that 6{z) = z. Then, 

• if u > 1, or V = 1 and 7 < 1, the system is always ergodic; 
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if V < \, or u = 1 and 7 > 1, there is a finite ergodicity hound M if, 



and only if, ^jyof^^fj < 00. 



Proof. The first step is to obtain a convenient estimate for An when n is 
large. Using the property < ai < a.j < 1, vahd for all i, one gets 



log An = Y, log = -7 E '"^ + ^(1) 



i=l i=i 

The finiteness of M depends strongly on the properties of the series ^ A„, 
which can be deduced from the following estimates. 

• If > 1, then the Dirichlet series X]j>i i~^ converges to some positive 
number, so that the series "Y^An diverges, and then clearly M = +00. 

• If 1/ = 1, then An ~ Cin~'^ . When 7 < 1, ^An still diverges and 
M = +00; conversely, when 7 > 1, 



00 



A, 

J i=j+l 



and M is finite whenever the second moment ^jy^j'^^j of the distri- 
bution A; > 1} exists. 



If < 1, then An = 0{e '^^ ). Using classical techniques (Riemann 
sums and integration by parts), one gets 



E e-^^'-' « / e-^^'-'dx 
i=j+i 



7(1 -z.) 



and hence M is finite whenever 'YjyQj'^^^^j is. 

This concludes the proof of the proposition, the results of which will be 
illustrated in Section [4.51 □ 



3.4 Picking the largest of c scores 

So far, all models which have been considered are based on exogenous pa- 
rameters ai, supposed to be known. Actually, it turns out that most of the 
computations remain valid when these parameters are a functional of the 
state r{t). We tackle hereafter such a model, where the ergodicity conditions 
can be obtained explicitly. 

The algorithm is simple: c > 1 POIs are selected uniformly, and the 
one with the largest score will be visited. This model is reminiscent of the 
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queuing model of which proves to be very efficient in thermodynamical 
Umit. The probabihty of selecting a POI with score less or equal to i is then 



1 



r(l) 



whence we get immediately 



7ri(r) 



r{lY 



(3.16) 



j=i i=i 

Prom a formal point of view, we are actually in the framework of Sec- 



tion 



3.3, as it can be seen by setting 



cr( 



' ^ .7 = 1 .7 = 1 ^ 



r(l) 
c 



(3.17) 



and remarking that the a!^s are no more exogenous constants as before. To 
check they form an increasing positive sequence bounded by 1, it suffices to 
note that x'^ is a convex function of x, for any c > 2, so that 



y-x 



choosing here 



X 



i i—1 

i=i j=i 



Vx < y < z, 



rj, z = r(l). 



While the equations are similar, the situation is nonetheless very different 
from Section 13.21 The model is more difficult to solve because 7r,'s do not 



have a simple expression in terms of the score i and a solution like (3.10) 
is not really usable; however, they can be easily computed by recurrence, 
since the normalization constant r(l) is given by (3.6). As a consequence, 
the ergodicity condition is easier to obtain. 

Theorem 3.5. Assume 9{z) = z and that the POIs are visited according to 
policy (3.16). Then, according to the definition given in Section^ the system 
is ergodic if, and only if, 



A 



Proof. Define 



Br, 



i=l 
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Then (3.9) takes the form 



which by a direct summation yields 



Rn+l — H Bn, 



(3.18) 



with Ri = X/ fi, Bi = 1 and B^ -Boo = '/''(I)- Since we are looking for a 
sequence satisfying, 



R„ 



lim i?„, = r(l), 



it will be convenient for the sake of consistency to rewrite (3.18) as 

_( a \ R^ X 



(3.19) 



remembering that r(l) = C-fC by (3.17). The interest of the form (3.19) is 



that it allows for a direct graphical analysis of the convergence problem, 
starting from the following simple facts. 

1. The curves {y = Ax'^+Bn} and {y = Ax'^ + Bn+i} are convex, parabolic 
shaped, and obtained from each other by translation of vector ^<I>„ 
along the y-axis. 

2. The factor cR'^^ is the value of the slope at the limit point when it 
exists. 



Hence one sees that (3.19) defines an increasing sequence converging to a 



unique value Roo, if and only if the expansion factor a/fiK is < 1. In this 
case, letting n — t- oo, we get immediately 



Roc = r(l) 



a + Xip'{l) 



which coincides (as expected !) with the result given in (3.6). To conclude 
the proof of the theorem, it suffices to quote the equivalence 



a 



< 1 



X - c-1' 



□ 
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3.5 About the general case: a matrix form solution 

The problem becomes more intricate when 6(z) is an arbitrary probabihty 
generating function with (/^'(l) < oo. Then the fohowing approach, based on 
the recursion (3.8), involves infinite matrices and seems to be computationally 
quite acceptable. We sketch hereafter its main lines. 



Clearly, equation (3.8) can be rewritten in the vector form 



n+l 



A, 



Un H Vn+l, 



(3.20) 



where 



• Un is the n-column vector [rnTn-i ■ ■ ■ rif'", where denotes the trans- 
pose operation; 

• T4i is the n-column vector [^*n-i5 0, . . . , 0]"^; 

• Mn{x) is a (n -|- 1) x n matrix for all x > 0. Its first row is the vector 
xBn, where Bn denotes the row vector 

and the remaining internal matrix is exactly the n x n identity matrix 
denoted by In- 

By a direct algebra, the solution takes the form 

, n+l 



u, 



n+l 



(3.21) 



i=l 



where 



Qn,ii^) = Mn{x)Mn-l{x) . . . Mi{x), i < U, 



^Qn,n+l{x) = In+1- 



The next step is to carry out the scalar product of (3.21 ) with the row vector 

dof 



A. 



n+l 



[O-n+l, 



,ai\ 



and then to let n — )• oo. Keeping in mind that K = X]j>]^ ai^j, we get the 
following final relationship, which is formally similar to the one given by 

dslTl), 



a 
A 



a 



oo ra+1 



An+iQi 



n.=0 1=1 



/ a 

4^ 



Vi- 



(3.22) 



From the definition of M„, each term of the double sum in (3.22) is clearly 
a decreasing scalar function of K. Hence, ergodicity conditions could be 
obtained along the same lines as in Theorem |3.3[ by analyzing the series in 
the right-hand side member of (3.22). 
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4 Some simple examples and limit cases 



For the sake of completeness, we quote hereafter simple peculiar cases, which 
shed light on some aspects of the behavior of the system. 

4.1 No creation of POIs, i.e. A = 

In this case, the system is empty at steady state for any Markovian policy. 
The argument is elementary, since the vector state is absorbing due to the 
M/M/oo character of the score decreasing process. 

4.2 The lower bound a = 

Here no POI will see any increase of score upon being visited, and hence one 



gets a clear lower bound for the model. Solving (3.5) leads to 



^ k>i ^ 

Consequently, when a 7^ 0, we have the following bound, valid for any point 
selection scheme (i.e. policy) vr, 

> Ak-(l) + 2^-(l)] 

and therefore the mean score r'(l)/r(l) of a POI can be finite only when 
(p"{l) is finite. This leads to some unintuitive behavior, since it was shown 



in Proposition 3.4 that having (p"{l) = 00 can at the same time make the 



system always ergodic. 

4.3 Score independent visits 

When the POIs are visited uniformly regardless of their score, which amounts 
to the choice 



we have vr(z) = r(z)/r(l). Then equation (3.5) allows to derive immediately 
the corresponding generating function, which takes the compact explicit form 

^ Ar(l)M.)-l) 
^ ^ Q(l-e(z))+/xr(l)(l-z"i)' 

where, for A / 0, 



and one can check the denominator of r(z) does not vanish for \z\ < 1. 



Moreover, r(l) is clearly finite and given by equation (3.6). So, we have 
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obtained a unique function r{z) analytic in the unit disc and continuous for 
1 2; I = 1. Consequently the system admits of a unique limit invariant measure 
{fiji > 1} for any value of the parameters. Intuitively, this is due to the fact 
that the decrease rate of the scores is somehow proportional to the number 
of points, like in a M/M /oo queue. But, when the selection process favors 
heavily enough the POIs having a big score, proper ergodicity conditions 



appear, as in the models analyzed in Section 3.2 



4.4 An unstable model with a score dependent policy 

A policy that may seem attractive at first sight is to give to each POI a 
weight proportional to its score 



7ri(r) 



so that 7r{z) = zr'{z)/r'{l) and (3.5) becomes a differential equation 

a [1 - e{z)] ^ + /.[I - = mz) - 1]. 

r (Ij 

However, the weight ai= i given to the score i is unbounded, and from 



Proposition 3.2 we conclude that no choice of parameters can lead to a stable 
system under this policy. More precisely, there exists at least one POI having 
an infinite score. 



4.5 A case study of Proposition 3.4 



Assuming the conditions of Proposition 3.4 hold, let us take in addition 
^{z) = z and, for some 7 > 0, 



I \ 7 



i + 1 



This implies = l/{k + 1)''' and therefore (3.10)-(3.11) become 



rk+i 



a 



A 1 /^iK\k 

Ji {k + Ip J ' 

00 

F(x) = ^Afcx'= = <I>(x,7,l)-l, 



fc=i 



where $ stands for the classical Lerch function [T, formula 9.550] 



$(x,7,i;) = ^ 



k=0 



(k + v) 
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— Y=2.1 

— Y= 1.7 
-7=1 
— 7=0.5 



Figure 1: Expected score of POIs per player as a function of the visit 

rate a. The parameters A = ^ = 3 remain fixed. 



When ^{z) = z, the computation of the expected score of POIs per player 
r'(l) is easy: 



00 ^ 00 



r'(l) = Y^kr^ = Y^-Y^kA^[^\ = - ^F' 



fc=i 



fc=i ^ fc=i 



a \ a ^1 f a 



The above formulas, allow to represent the expected score per player as a 
function of a in a parametric way, by letting x vary from to 1 (Figure [T]). 
As expected from the theoretical results obtained above, 3 different situations 
prevail: 

• 7 < 1: the system is always ergodic (dashed curves); 

• 1 < 7 < 2: there is an explicit ergodicity condition on a, and r'(l) 
tends to infinity at the boundary; 



7 > 2: there is an explicit ergodicity condition on a, but r'(l) remains 
bounded. 



5 Miscellaneous outcomes and open problems 

The next three paragraphs present briefly some ideas and mathematical 
questions concerning the general model. 



19 



5.1 About a general solution via an integral equation 



3.2 



and let a(z) = X^,>x ajz^ . Then by 



Consider the situation of Section 
Hadamard's theorem |7j and definition (3.7), we can write 

z \ dw 



7r{z) 



1 

2iiTK 



r{w)a 



which yields the non-hnear integral equation 
fir{z) 



a[l-e{z)] 
2i7TK{l - z) 



, , ,z\Aw A(l — 09(2;)) ,^ 
r u; a - — + \ " . 5.1 
^ w J w 1 — z 



Methods of solution of (5.1) (e.g. via boundary value problem or Fredholm 



integral equation) depend heavily on the explicit form of a(-). Alas, tractable 
explicit expressions can hardly be derived (nor expected!), even when a(-) is 
a polynomial. 



5.2 About transient phenomena 

Hereafter, we do not pretend to give exhaustive rigorous proofs of our claims, 
which can rather be viewed as very likely true conjectures, and have been 
verified by several numerical simulation runs. 



Our concern is to extend Theorem 3.3 when there is no solution to (3.15 ), 
i.e. when its right-hand side member is finite and a is too large. In fact, it 
appears that the system has still a stationary regime, but a finite number of 
POIs have an infinite score. This means that, as t — )• 00, the selection policy 
7r(t) becomes defective, namely (see equation (3.1)) when 



TT < 1. 



This implies exactly 



vri(f) 



airi 



where J is a positive constant representing the proportion of POIs going to 
infinity. Obviously, 5 = when the system is normally ergodic, i.e. no escape 



of mass to infinity. Then the consistency equation (3.11 ) must be modified 
and becomes 



1 



-F{—]+- 



(5.2) 



which has always a solution {K, S) by monotonicity with respect to K, 
remembering that F(l) < 00, while F(l + e) = 00, Ve > 0, and a > XF(1). 



Equation (5.2) is not sufficient to determine the two unknowns K and 5. To 



get a second equation, we resort to the following heuristic stochastic least 
action principle, which in our opinion should be provable in a very general 
context. 
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Definition 5.1. The least action principle. When a muticomponent 
irreducible Markovian system ceases to be ergodic, the first component(s) 
becoming infinite with positive probability) can be identified by continuity with 
respect to the set of parameters V defining the system. 

This definition, which might look obscure to the reader, can be rephrased 
by saying that the phase transition takes place as soon as one touches some 
regions (hyperplanes or surfaces) in V. This is for instance the case for the 
famous Jackson-Kelly stochastic queuing networks. . . 

Applying this principle leads to say that we are looking for the minimal 
5 > ensuring the system 

r = lim r(t) 

t—^oo 

is no more ergodic, so that some POIs go to infinity. This implies necessarily 



a 



1. 



(5.3) 



Then (5.2) and (5.3) yield at once 



, a-AF(l) 
c> = , 

K-6 XF(1) ^ 
vf = — -— = — < 1, 



K 

A, 



a 



r(l) = -[F(l) + y,'(l)]. 



A possible way to decide analytically when vr < 1 would be to design a 



converging iterative scheme to solve system (2.2) or (2.4) (see e.g. the 



polling network analyzed in [Ij): this appears to be another interesting, but 
practically intricate problem. 



5.3 The case ^'(l).^'(l) = oo 

When either ^'(1) or (/^'(l) are infinite, we are a priori facing a transient 
phenomenon as r(l) = oo. However, the system can have an interesting 
behavior provided that K = X]i>i '^i^i < oo, which yields necessarily 

lim inf = 0, 

but this would correspond to a non-herding system, which is an other story, 
although most of the mathematical arguments presented in Section [3] could 
be readily applied. 
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A Generators, cores and weak convergence 



Here we quote the material necessary for the proof of Theorem 2.2 The 
results are borrowed from [2]. 

Theorem A.l. If O is the generator of a strongly continuous semigroup 
{T{t)} on C, then its domain T>{0) is dense in C and O is closed. 

Definition A. 2. (J^, p. 17) Let O he a closed linear operator with domain 
T)[0). A suhspace S ofT>{0) is said to he a core for O if the closure of the 
restriction of O to S is equal to O, i.e., if 0\s = O. 

The next proposition ([2j, p. 17), is an important criterion to characterize 
a core. 

Theorem A. 3. Let O he the generator of a strongly continuous contraction 
semigroup {r(t)} on C Let Dq and D be dense suhspaces of C with Dq C 
V C V{0). IfT{t) : Do^V for all > 0, then V is a core for O. 



The proof of Theorem 2.1 relies heavily on the next general proposition 
([2], theorem 6.1, p. 28). 

Theorem A. 4. In addition to C, let Ck,k > 1, be a sequence of Banach 
spaces, Ilfc : C ^ Ck he a hounded linear transformation, subject to the 
constraint sup \\Ilk\\ < co. Let also {Tfc(t)} and{T{t)} he strongly continuous 
contraction semigroups on Ck and C with generators Ok and O. We write 
fk^fto mean exactly 

f£C, fk€Ck iovk>l, and lim ll/fc-nfc/ll =0. 

Then, if D is a core for O, the following are equivalent: 

(a) For each f £ C, Tk{t)Ilkf T{'t)f for all t > 0, uniformly on hounded 
intervals. 

(b) For each f e C, Tk{t)Ukf T{t)f for all t > 0. 

(c) For each f £ D, there exists fk G T>{Ok) for each k > 1, such that 
fk^f and Okfk Of. 
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